Vision-Based Control of a Mobile Base and On-Board Arm

In this paper we present a new methodology to achieve visionbased control using a mobile manipulator to reach a required threedimensional target position and orientation. Examples of mobile manipulators include standard forklifts and backhoes, which have a fixed manipulator mounted on a mobile platform. The overall objective of this methodology is to apply the nonholonomic degrees of freedom represented by two independently driven wheels together with the holonomic dexterity of the on-board arm in order to bring about a desired positioning objective for the arm. Mobile Camera-Space Manipulation and an extended Kalman filter are implemented to use a minimum number of holonomic and nonholonomic degrees of freedom as required for the end member of the on-board arm to achieve a target position and orientation. We present this vision-based method to control mobile manipulators as an alternative to methods such as visual servoing and calibration (i.e., stereo approaches). This new approach only requires a minimum number of holonomic degrees of freedom to achieve the required three-dimensional positioning by taking full advantage of the nonholonomic degrees of freedom. Experimentally, the method shows itself to be robust to errors in the nominal kinematics and the optics models, as demonstrated by the